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ABSTRACT 



py tracing explicit the implicit assumptions about 
proura-i costs and benefits, r<>sojrcp allocation in a nonprofit 
organization could be simplified. ft rational ^cheite for i mplerent inn 
such a proorar nioht beain by usina detailed interviews with prooram 
administrators to determine input and output scales. r ol\oving this, 
respondents could choose among feasible inputs for each of a series 
of budgets, ar.d these inputs could then be compared with the output 
scales to measure each respondent's assumptions about the level o f 
output possible from the level and type of ir.ru*. Such information is 
useful in analyzing potential changes in resource allocations. ( r M 
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IDENTIFYING EFFECT OF POTENTIAL 
BUDGETS ON lVtOGNAII OUTPUT 

The difficulty in measuring costs and benefits, especially in nonprofit 
organizations) docs not, of course, prevent analysis of resource allocation within 
this framework. In fact, such analyses are routinely made through implicit 
assumptions about cost3 and benefits and their relationships. This paper proposes 
a procedure for maxing these assumptions explicit. 

In Part I the paper will describe a method for developing input and output 
(or performance) scales; for identifying tha trade-off between inputs for several 
levels of activity through identifying points on «.he Engel curve; and for measuring 
the effect on program output of the points Identified on the Engel curve. Part 11 
will repor( on two applications of the teciiniquo. 

PART I 

COST BENEFIT ANALYSIS 

As with any cost benefit analysis, the technique proposed identifies the 
effect on output of different input levels. It Joes so through identifying for 
a series of budgets preferred combinations of inputs allowed by each budget and 
comparing these preferences with output a3 represented by a scale. If one were 
relatively free to tra!e off among three inputs, then, for each of a series of 
budgets, he could indicate that input mix which he thinks would best accomplish 
the goals of the progran--at least as he sees them. If one tnen had an output 
scale, each of these preferred input combinations could be compared wit* that scale 
to estimate what Output could be achieved. 

Ideally, tuts analysis should be done through identifying production indifferent- 

curves (transformation curves or isoquants);* consumption indifference curves; 
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and the Engel curve (consumption-income graph) as described by points of tangency 
of successive production and consumption Indifference curves as resources Inputs 
are increased. This would mean identifying the curves or functions in Chart t 



which illustrates a series of indif feronce curves and four tangency points on the 
Engel curve A, d, C, and L». fue a and V uxe3 represent two input3 into the pro- 
gram and the Engel curve indicates the preferred pattern of inputs. As illustrated 
in Chart II, the Engel curve is then related to output indicating the level of 

output to be expected from successive points on the Engel curve which, of course, 
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reflects levels of input. This approach differs from the ideal cost-utility 
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analysis which seeks to relate input to output through a production function. 

In the approach being set forth here the production function is implied by the 
trade-off of inputs as expressed through the Engel curve. Obviously this is a 
consumption approach to the analysis. 

Identifying all of the functions Illustrated in Chart 1 would be very 
difficult In most situations. An alternative to doing so is simply to identify 
a terles of tangency points. This would give some information on the location of 
the Lngel curve and allow a comparison with output. Assuming that the points of 
tangency are consistent with the technical limitations on the use of resources 
and with the Indifference consumptions function, marginal productivity and mar- 
ginal utility of the various allocations would be reflected. 

DEVELOPING OUTPUT SCALE OR OBJlCTIVKS OF PAOCHAM 

While It Is difficult to Identify output of programs such as hospitals, 
libraries, and research and development, It can be approximated and in practice 
is done through crude surrogates. An underlying assumption of the output scale 
proposed here Is that In virtually any process of evaluating resource allocation 
some empirical referents for output are used whether or not their usefulness Is 
empirically or logically established. These referents could be Identified through 
detailed Interviews with appropriate people to provide the basis for a scale. The 
Interview could secure Information on how these Individuals view the mission of 
the program and the referents they use In the Identifying performance of the pro- 
gram. The emphasis in the interview would be to secure empirically identifiable 
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phenomena, "proximate criteria, ** used by these individuals it. considering their 
program's operation . J 

Using this information it would then bo possible to describe different, levels 
of output moving from a low point to a high point of quality as viewed by the 
interviewee. Possibly the best way to do t’.ii3 would be to prepare a rather 
detailed description for several levels which will clearly be mutually exclusive. 

One could in this way construct a unidimensional, partially ordered scale. One 
night describe several aspects for each level making sure that each aspect 
described i3 clearly different from the same aspect on other levels. One would 
have the following arrangements 

Output Scale 

Aspect 1 Aspect 2 Aspect 3 Aspect 4 Aspect m 

Level 1 
Level 2 
bevel 3 
Level A 
Level n 

In this table each cell in each column represents a detailed description 
which will be different and mutually exclusive from the items above and below 
(so&o aspects, of course, might not be applicable at all levels). This will 
Insure that each level la clearly not overlapping with any other. 

In preparing an output scale for a park's picnic facilities, one could use 
three aspects: condition of the facilities, quality of the facilities, and 

user satisfaction. For each level one can then write a description for each aspect. 
For the highest level the descriptions would indicate the very best situation 
where the facilities wore of the best quality, kept in the test of care, and net 
all the needs of toe users. As one noved down the scale, one would have three 
descriptions for each level of a situation which vas less desirable than the one 
O 
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above it, but all descriptions would be clearly not overlapping. If based on an 
interview, the descriptions should reflect as closely as possible the empirical 
referents of the respondent. One could proceed to develop a scale for other 
programs of the park or one for the park itself. 

The validity of such an output scale would be no greater than the validity 
of the referents themselves. Such a scale, if nothing else, however, would at 
least make explicit the empirical evidence being used to identify output of the -• 
program. Assuming It has some reasonable validity reflecting the experience and 
understanding of the individual or individuals from whom the data were secured, 
one would have some empirical information to use in considering resource allocation. 
IDENTIFYING POINTS 0.1 THE ENGEL CURVE 

To estimate points on the Engel curve one needs to prepare a scale for each 
input. This can be done as it was done for the output scale. One can prepare a 
set of descriptions for each input, proceeding to describe aspects for each input 
for as many levels as seems appropriate. In the case of an input such as personnel, 
for example, one might have described several levelB of three aspects-^expericnce, 
education and number. (In a matrix as illustrated in the previous section, one 
would title it "personnel" instead of "output."). While one would have only one 
matrix for output, he would have one for each input. 

Next one would secure cost data to estimate the erst for each level of each 
input. Using this information one can then prepare a questionnaire based on a 
series of budgets which for the respondent would only be identified by a number 
or letter. The questionnaire would indicate t Ire different combinations of inputs, 
in terms of levels, that would be possible for each budget. The quest iotmaire 
might appear something like the following for a particular budget. In this 
example, the respondent would be asked to select configuration 1, 2, 3, or 4, and 
thus decide which of his three Inputs would be at level three and which at level 
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four. Similar options would be given for all other budgets. The number of con- 
figurations will typically incrcosc as the budget Increases. 





Configuration 1 


budget A 
Configuration 2 


Configuration 3 


Configuration A 


Input 1 


Level A 


Level A 


Level A 


Level 3 


Input 2 


Level A 


Level A 


Level 3 


Level A 


Input 3 


Level A 


Level 3 


Level A 


Level A 



In going over the choices available with each budget, the respondent must 
consider the trade-off between t lie various resource inputs. The questionnaire 
itself presents for each budget several points on a production curve or Isoquant 
and hence the respondent's choice will represent that point closest to a tangcncy 
point on his preference curve, a point on the Engel curve. Since each budget 
represents only selected points on the Isoquant, the respondent's choice may not 
precisely represent a tangency point but tne budget conf iguration closest to a 
tangency point, ily increasing budget choices onu can come closer to the true 
point on the Engel curve. A second questionnaire could be prepared once the 
general area of tangency has been located. 

In many, If not most, situations It will be quite difficult to estimate with 
a high degree of accuracy the relationship of each budget to the various ' rr>ut 
levels. One would in effect bo working with production indifference curves that 
would have to be thought of as wise curves with the width indicating the error in 
the estimate. 

RELATING ENGEL CURVE TO OUTPUT 

Identifying points on the Engel curve gives information on a person's pre- 
ference pattern or trade-off among available inputs but does not indicate what 
output levels can be achieved. It would, therefore, be very helpful to know 
something of the relationship between the points on the Engel curve and the out- 
put scale. 
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Assuming that the respondent would like his choice behavior at least in 
resource allocation to be consistent with tiic axions of utility theory^ in that 
his choices will be consistent and transitive, then one can use the procedure 

g 

described by von Neuman and .lorgenstern as well as by others in later works to 
measure the utility of each selected input configuration (estimated points on 
Engel curve) In terms of each point on the output scale. These input configurations 
are the ones selected by respondent through the procedure described In the 
immediately preceding section. Using such a lottery or gaming device will allow 
one to construct an Index comparing the respondent's input making explicit the 
effect of each configuration on output that the respondent expects. 

The procedure allows one to measure tne respondent's expected utility for 
each input configuration in terms of performance as represented by the points on 
the scale with which it is being compared. The respondent 19 in effect being 
asked to weigh in his mind the output he thinks he can secure from a given input 
configuration with a given chance at a certain output. He must decide if the out- 
put he thinks he can secure from a given input configuration either exceeds or 
is less than (say) a 5UX chance of achieving a specified outputi and at some point 
be Indifferent vaere lie has a choice between 100/E chance of receiving a given 
input and a 101 chance as point lit on the scale. This would reflect a low 
utility for that input configuration compared with point 111 on the scale. 

It Is, of course, Important to reemphasize that the cost estimates be 
sufficient accurate to give the respondent realistic choices. Furthermore, the 
input and output descriptions must be consistent with the respondent's concept 
of input and output and of program goals. This will reduce his uncertainty about 
the context within which he i3 to answer and will more accurately reflect his 
assumptions. 

The respondent's indifference points could be displayed in the following 
manner with the entires In the cells Indicating point of Indifference. 

O 
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Budget Configuration Output Scale ’./here Each Point (Level) 

Proferi nces (Points o.i Equals Utility of One 

Engel curve) 





i 


II 


III 


IV 


Budget A, 


.05 


.25 


.60 


.90 


Budget b 3 


.15 


.30 


. OO 


.100 


Budget C, 


.20 


.35 


.05 


.100 



Each percentage indicates the point at vhich the ri.rpondent io 
indifferent bolween that opportunity ta achieve the level at the herd of the 
coin cm xi.iJ a 103X chance to receive tue budget for that row. JnJer the 
second coluPu, for Budget 3^ Ilia respondent i3 indifferent between a 3*) 
percent cnancc at Level II and a 1JJ percent chance at the budget. Above 
30 percent ne would prefer the level but below 30 percent ne would prefer 
the wu,: v ,et. the figures for each budget indicate what the respondent thinks 
of t*> otential of each budget in terns of each level. The higher the 
luvei toe lower t.ie valae of each budget in terns of that level. 

could conpare each budget choice only with level one on the output 
Sts- ng just the first colunn. By conparing, hovever, with each point 

on f .tout scale, one has additional inf or tm ion to analyse the output 
scale and to secure sooe Insight into the respondent's S3sucption about toe 
aarginal returns as inputs are increased. 

UTlLtZthJ TnE fECiIUlQUE 

Tne principal value of the technique is making explicit people's 
assumptions about the relationship of resource Inputs and eapirlcal inilcants 
for a program including assumptions awout incremental changes in resource 
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allocations. Through making the assumptions explicit trie technique provides 
a basis for analyzing the program and for discussion by Interested parties 
of resource allocation to the program. If tite technique were u3ed to secure 
the assumptions of several people concerning the program, this information 
could be used to clearly identify difference of opinion and provide a basis 
for developing a concensus concerning the effect of resources on the program. 

If tne technique were used for several programs, it would provide assistance 
in comparing the relative effect on the programs of increases in resource 
allocation. 

The technique can also be used to analyze the logic of the assumptions 
being made in resource uses. Inconsistencies could be exposed, for example, 
between trade-off expressed in tne Engel curve and the output scale. 

Even if the entire technique is not used, aspects of it can still be 
of use. the suggested scale for identifying output for example can be used 
simply to make explicit what the empirical referents are for identifying the 
output of a program which in turn will focus attention on being explicit 
about tne purposes of the program. It will identify the referents used by 
tne respondent in the preparation of budget recommendations and thereby 
provide a basis for reviewing their usefulness as proxies or surrogates for 
tne objectives of tne program. As long as the referents are not made explicit, 
such a reviev/ might not be easily made. One also has the basis for a review 
of tne program objectives. 

The output scale can provide an alternative to the use of a single factor 
in the evaluation of a program. So often an easily identified 3 ingle factor 
is u3ed because of it3 availability even though the rationale is \;eak. For 
example, tne number of books in a library is typically used as a measure to 
give some indication of the usefulness of a library or the effect of resource 
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allocation to the library. In neither case is the number of volumes very 
useful because it leaves -.unsaid the quality of the books, the appropriateness 
of the books, and Cue quality of library service. 

Another possible use of the output scale r .;hen used without the other 
elements of the technique discussed would be as an alternative to the typical 
statement of goals and objectives which are developed to guide the performance 
and evaluation of a program. These statements are usually not operational 
and where they are, they reflect only one level of operation or achievement. 

In providing several possible levels of achievement in empirically identifiable 
terms, an administrator is in a much better position to conceptualize the 
possible achievement of his program. 

Where one can identify points on the Engel curve, he can estimate roughly 
the shape of the curve. This provides information on the trade-off betvreen 
input and output for different levels of resources. The curve provides the 
basis for predicting what a person nay do if resources available to him were 
changed. The curve, by making explicit his preferences in resource use, 
provides the basis for discussion with others concerning the use of resources 
in the program. If the curve were developed for eacn of several people, then 
the information could be used to reach a concensus on resource use. 

APPLICATIONS OP £UE TEChNI^UE 

Tnis technique can be used to make explicit the assumptions of people 
regarding the effect of resource allocation for a variety of types of programs 
where output is not clearly defined as a specific product or group of products. 
It coulc be used in the analysis of programs administered by municipal govern- 
ment to provide recreation, police, fire, garbage collection, or street 
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maintenance than seeking some form of unit cost per product or service 




one could use this technique which could provide much better information. It 
would not be as satisfactory as a unit cost study that a manufacturer might 
compute but would be an improvement over vague assumptions and guesses that 
might otherwise be used. For example, developing the output scale for a parks 
program would make explicit the purpose of a park and the referents for 
identifying the degree to which the purpose is accomplished. The complete 
analysis would make explicit the assumptions of people involved in the 
administration of the park and help to identify differing assumptions that may 
exist among them. In making the assumptions of all these people explicit, it 
would focus attention on the analysis of these assumptions. Using this technique 
to analyze each of the city's programs would make possible comparing resource 
allocation among the various programs. 

Similarly, the technique could be used at state and national levels for 
analyzing programs. It could be used in analyzing sections of a program as 
well as total programs, If the technique were used to analyze very large 
programs, it would, of course, become an expansive task, Developing input and 
output scales for a national program in parks and recreation and securing 
cost data would be a formidable undertaking. Yet extensive resources are 
allocated based on assumptions made by various officials concerning the effect 
of funds on programs and making these assumptions explicit by this technique 
would be a useful service. 



limil the analysis to subprograms. This technique could be used in connection 
with program budgeting whereby it would be applied to each of the identified 



The alternative would be to 
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program of an agency for which separate budgets and economic analyses were 
needed. 

In a business firm there are possibilities for using the technique where 
funds are expended for programs who?:, output is difficult to define. Some 
aspects of public relations programs suggest opportunities. Other opportunities 
might be found in programs for research and information gathering and f or 
programs granting funds to educational and research agencies. 

In a university the technique could be used to analyze the programs of 
academic departments. Identifying output referents for academic departments 
through a scale would alone provide very useful information. 

LIhl'fATIOdS OF TUL TfcCuJIQUE 

Ihe obvious limitation of the technique is tnat the most it may do is 
make explicit people's assumptions. It does not in itself establish the 
validity of trie output scale or of the relationship between input and output. 

It can, of course, provide a basis for doing this. The technique does not 
provide much information on the respondent's assumptions about the production 
and indifference curves other than as they are reflected in points to identify 
the angel curve. 

The value of the technique relies heavily on the ability to secure from 
a respondent information on nis assumptions aoout the effect of resource use 
and/or the empirical criteria that he uses in evaluating program performance. 
Such information will be difficult to secure where the respondent has not 
given nuen taought to the empirical aspects of his program's performance or 
his assumption about relationship of resource input to resource output. Where 
he has he may 3till nave difficulty expressing himself on these points. 
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The technique relies on the availability of people well acquainted with 
the progvam. Only people well acquainted with the program can le assumed to 
have formed useful ideas about output of the program and the effect of resource 
allocation. The technique also depends upon the respondent's behavior in 
adhering to the axioms of utility theory and the respondent's ability to 
respond to choices stated in terms of probability. 

Tne technique can probably be used most easily with relatively small 
programs. As the program becomes largef, It may be more difficult to secure 
the necessary information and yet as programs become larger, it is more vital 
to make explicit people's assumptions about the program. 

Tne technique, of course, suffers all of the limitations associated with 
securing information from people about their opinions through interviewing, 
such as the problems of securing cooperation, candidness, understanding inter- 
view questions, and all the others. The use of the technique requires a 
skillful investigator . 
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PART II 



EMPIRICAL EXPERIENCE WITH TilE TECHNIQUE: A HOSPITAL FOOD SERVI 3E 

The technique was tried in two situations: the food service and 

the pharmacy of a small hospital . 

DESCRIBING FOOD SERVICE INPUTS AND OUTPUTS 

A detailed interview was conducted with the individual directly in 
charge of the food service who administers its budget and originates 
recommended changes in that budget. An open ended questionnaire was 
used to guide the interview which was conducted by two people, one 
carrying the burden of the interview and the other recording the 
responses , 

There v/as no particular trouble securing detailed descriptions of 
the inputs which were readily and easily identified empirically. Securing 
a description of output was a little more difficult. As might be imagined 
the first description of output was in rather general and nonenpirical terms. 
One of the key questions which helped to focus the respondent^ attention 
on the empirical aspects of output or performance v/as to ask what specific 
items would be noted if asked to evaluate the performance of another 
hospital food service with a view toward making recommendations for change. 
Estimating Points on the Engel Curve 

Based on the interviews, the inputs were analyzed and grouped into 
four categories: supplies, personnel, space, and equipment. Four levels 

for each of these categories were described moving from a situation of 
rather meager resources to one with quite ample resources. For each 
category several aspects were described to reflect, for example, both quantity 
and quality. The descriptions were as empirically oriented as possible. 




Jnce the descriptions were completed, they were submitted to the. 
respondent to insure that they were consistent with his frame of 
reference for describing inputs into the food service program. The 
respondent reviewed the material, raised questions, and made a few 
suggestions. The descriptions were revised to reflect suggestions and elim- 
inate misunderstandings. 

Data were secured to determine the cost function for each of the 
four categories in order to estimate the cost of the inputs for each of 
the four levels described. There was no particular problem in securing 
the cost data with the exception of space which involved building 
depreciation. The objective in securing data was to insure that the 
questionnaire to identify preferences and the points on the Engel curve 
would be realistic for the respondent — go that the choices would be 
consistent with his experience and to be typical choices made in the 
administration of the food service as allowed by input cost3. 

Using this cost data, the questionnaire was prepared indicating the 
alternative input mixes possible for a set of budgets. The respondent 
was not told the amount of each budget. The budgets were identified 
only with a letter so that the respondent received a questionnaire with 
twenty-seven budgets identified by letter and an indication of the 
various combinations of inputs possible for each budget moving from a low 
to a high budget. The respondent was allowed to keep the description 
of the inputs while selecting his preference for each level. This gave 
twenty-seven estimated points on the ^ngel curve. It is assumed that 
the points did not fall on a smooth curve because of the lumpiness with which 
some of the resource inputs caue (personnel, equipment, space), 



' /& ~ 



Preparing the Output Scale 

Again based on the interview a four point output scale was prepared 
with a detailed description prepared for each point with several aspects 
described. It is shown in Addendum A. Again* as with the description of 
inputs, the respondent was asked to review the output descriptions and to 
raise questions and make suggestions. The output description was then 
revised. 

Identifying Relationship Beti/een Inputs and Outputs 

Using the technique described in Part I for measuring utility, 
the respondent was asked to make a choice between input configuration 
selected from those allowed by the smallest budget configuration chosen and 
the lowest of the four points on the output scale. The respondent was told 
that he had a fifty percent chance of achieving Level IV output or a 100% 
chance of receiving the particular input configuration involved. The odds 
were then varied to estimate the point of indifference, changing the odds 
in steps of ten. If the respondent preferred the budget even though the odds 
of receiving the given output level configuration reached a 100%, It was, 
of course, assumed that the respondent felt quite sure that with the given 
input configuration he could more than achieve the given level of output being 
compared . 

Once a point of indifference was found, the interviewer went to the 
next budget 3nd input choice. The interviewing procedure sought to place 
each budget choice somewhere between two points on the scale or below or 
above the end points. In the interviewing process if when asked the first 
question (comparing a 100% chance to receive a budget with a 30% chance 
to receive a given level) the respondent selected the budget, then the 
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chance to receive the level was increased in step3 of ten until indifference 
was found. If he selected the level then the odds to receive the level were 
decreased by ten until indifference was found. The results of the interview* 
ing are shown in fable I and Chart IV, 

la the case of budgets A and 3 the respondent preferred the budget even 
though his chance to receive Level IV output v;as set at 100/.. This meant 
that the output stated for Level IV was so low that the respondent felt that 
with budgets A and li lie could achieve at least Level IV if not a little taore 
than that. Since the respondent was indifferent between 100£i chance at Level 
IV and a 100% chance at budget B it wa3 a33umed that the point of indifference 
for C would be above Level IV. As the chart shows, he preferred C until the 
chance to receive Level III dropped to .20. In the case of budget G he 
preferred the budget even where he had a 100% chance to receive Level III. G 
was then compared with If and the point of iruiif ference was .10. This infor- 
mation would indicate that the respondent felt he could achieve Level III and 
possibly something beyond that. In some cases budget increments v/ere so 
small or had so little effect on critical resource choices that respondent saw 
little difference. See, for example, choices for budgets ?, Q, and R. He nay, 
for example, have been more interested in increasing personnel which required 
a fairly large budget increase than increasing supplies made possible by small 
increments, budgets V, W, and X took program above Level I, As the budgets 
are increased one has some insight into what the respondent thinks can be 
achieved witn the budgets. The comparisons ore rough but they do give an 
insight into the respondent's thinking and express his assumptions quantita- 
tively , 
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POUT OF INDIFFERENCE KEIWEEU A SERIES OF DUDGETS 
AID OUTPUT LEVELS OF FOOD SERVICE 







Point of 


Indifference Levels 




Budget 


I 


II 


III 


IV 


A 








• 100 


3 








100 


c 






.20 


100 


D 






.40 


100 


C 






.40 


100 


F 






.40 


100 


G 




.10 


100 


100 


H 




.10 


100 


100 


I 




.40 


100 


100 


J 




.40 


100 


100 


it 




.40 


100 


100 


L 




.30 


100 


100 


a 




.30 


100 


100 


a 




100 


100 


100 


0 




.80 


100 


100 


p 


.40 


100 


100 


100 


1 


.40 


100 


100 


100 


K 


.40 


100 


100 


100 


S 


.00 


100 


100 


100 


T 


.30 


100 


100 


100 


U 


.30 


100 


100 


100 


V 


100 


100 


100 


100 


w 


100 


100 


100 


100 


X 


100 


100 


100 


100 




INDIFFERENCE POINTS BETWEEN A SERIES OF BUDGETS 
AND OUTPUT LEVELS OF FOOD SERVICE 
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In Chart IV the points on the X axis are equidistant since the X 
axis represents the budget and budget increments were equal, The 
distance between the four points on the Y axis, however, is not 
necessarily the same. The chart is drawn to assume this but that Is 
not necessarily the case. The detailed descriptions assume they arc mutually 
exclusive but that is all. The interviews with the respondent suggested that 
the distance between III and IV was considerably greater than between II and 
III and that the distance between II and III was somewhat greater than I 
and II, but not very much. The distance between the four points was 
not measured but the next test of the technique explored this problem. 

EMPIRICAL EXPERIENCE WITH THE TECHNIQUE IN A HOSPITAL PHARMACY 

The procedure followed with the pharmacy was essentially the same as 
that used in the food service with a few exceptions. The output scale 
(see Appendix B) had five instead of four points to make possible the use 
of a device described below to secure information on the ilstance between 
points on the scale. In input, :: space 11 was riot used because it was not 
a critical factor as in the food service and because of problems in 
securing meaningful cost estimates. The total number of budgets used in 
estimating points on the Engel scale was reduced since there was not the 
resource lumpiness found in the other operation. The interviews to 
secure data for the descriptions were taped because it became quite 
evident in the fir3t interview that a complete transcript of the 
respondent comments was important if not critical in preparing the descriptions, 
The principal difference from the technique used in the food service was 
in the procedure to compare inputs with outputs. Instead of Ju3t one 
interview several were conducted to test alternative approaches to the one 

used in the food service study. 

O 
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First Interview to Identify Relationship Betveen Inputs and Outputs 

Using the technique descr-bad in Part 1 to censure utility, the 
respondent was asked to compare each budgetary choice with each output 
level. A set of cards was prepared which identified the budget and 
listed cne configuration of inputs which the respondent had selected 
for that budget, The respondent wa3 then told that ho had to choose 
between a fifty percent chance to receive that budget or a lOOh chance 
at level V. When a point of indifference was found or the respondent 
did not change from iiis first preference, he was then qusriod in a 
similar manner but asked to compare the given budget with level IV, 
then 111 and so on until liis points of indifference was found between 
that budget and each of the five points on the output scale. 

The results of the interview are shown in Table 11. 



TABU 11 

Point of Indifference Where Utility 
of bach Ley-'l Bag»d on Each Budget 

OUTPUT 



Budget 


I 




111 


IV . 


A 


0 


0 


0 


0 


6 


0 


.70 


.25 


.60 


C 


.CO 


.50 


.05 


.35 


a 


.70 


.90 


.40 


.30 


L 


.CO 


.SO 


.40 


.30 


f 


.50 


.90 


.45 


.30 


0 


.70 


.90 


.00 


.20 


A 


.70 


.00 


.30 


.30 


i 


.SO 


.70 


.20 


.30 


J 


.70 


.70 


.30 


.30 


R 


.so 


.70 


.40 


.30 


L 


.90 


.70 


.20 


.30 
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Since in the interview, the steps in the questioning when 
changing tho odds were 10% at a time (I>0% chance, 4C'» chance, and so 
on) the point of indifference would be between tho figure in the table 
and ten percent above. In the column under V, the points of indifference 
would be between 0 and 10%. 

This table in effect compares each level of output with each 
budget, so that in reading from the right to the left one gets a measure 
of the utility of each point on the output scale in terms of the utility 
of the budget for tho row being read, for example, level V clearly 
has a very low level of utility in terms of any budget. This is the re- 
verse of what was done in the food service test. 

This tabic was prepared to give some insight into the difference 
between the five points on the scale which it docs. The distance be- 
tween IV and V seems to be greater than between 111 and IV. The space 
between II and 111 seems greater than the previous two while the space 
between I and II seems not as great aa any of the rest. Cnc could 
develop measures of tho distance using this data. 

Seco nd Interview to Identify Rela tionshi p s Between Input and Output 

In the second interview the process wa9 reversed from the first. 

The respondent was asked to cocpara each budget choice with each level 
but was told that he had a SOX chance at the level and a 100H eharcc at 
the budget. ihi9 technique gave the information that vas secured 
from the respondent in the case of the food s_rvice plus a great 
deal more. 

The results of this interview are given in Table HI (these not 
in parenthesis). In this case the table can be interpreted a9 giving 





the utility of each budget in terms of level of output. As one 
icads down column one he, in effect, reads the utility of each budget 
in terms of Level 1 of output. Where there is c .002 this means that 
that level or one higher could be achieved with the given budget. 

As one will note, there arc some odd figures in Table III. In 
column I, budget A, for example, is given a rating of 80 vhich seems 
oca. Because of such items, the respondent vaa asked to go over the 
table and insert in parenthesis his astinato of what he thought he 
could do with each budget in terms of each level of output. Some in- 
consistencies remained. For example, with budget 0 and E it would 
appear that if the budgets would allow a 50 and 30 percent chance 
respectively of achieving level II, they should allow at least that 
with level 111 rather than zero as shown. 

The tabic indicates that the respondent feels that with any 
budget he can at least achieve level IV. There is some differentia- 
tion between vhat he can do in achieving levels 1, II, and HI but 
not a great deal. 

It was felt that the redundancy of information in Table 3 would 
help to overcome problems of communication with respondent a9 veil 
as his ability to analyse decisions within the context of the standard 
gamble in measuring utility. It is felt that the table does do this 
in giving & reasonable estimate of vhat can be accomplished. Some of 
the apparent inconsistence is due to the estimate of the Fngot choices 
and some situations created by different budgets when cosoared with 
each other. This was true In both interviews and because of the 
problem the additional estimates are helpful. 
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TABLE III 

LEVEL OF OUTPUT POSSIBLE WITH A-L BUDGET IOTUTS 



BUDGET 


I I 


11 11 


III III 


IV IV 


V V 


A 


[0)80 


(0)70 


(0)70 


*(100)100 


(100)100 


B 


[0)70 


*(5)70 


(0)60 


(100)100 


(100)100 


C 


[10)00 


[75)90 


*(90)100 


(100)100 


(ICO) 100 


I> 


[0)70 


*[50)60 


(0)100 


(100)100 


(100)100 


E 


(0)60 


*[30)80 


[0)60 


(100)100 


(100)100 


F 


[0)70 


*[30)80 


(0)70 


[100)100 


(100)100 


G 


*[80)80 


(100)90 


(100)100 


(100)100 


(100)100 


H 


*[85)80 


(100)90 


(100)100 


(100)100 


(100)100 


I 


*(70)70 


[90)90 


(100)100 


(100)100 


(100)100 


J 


*(£>0)90 


[100)90 


(100)100 


(100)100 


(100)100 


K 


*(95)90 


1100)70 


(100)100 


(100)100 


(100)100 


L 


*[100)90 


(100)90 


(100)90 


(100)100 


(100)100 


The 


use of the 


pharmacy was probably not 


a very good 


choice be* 


cause tho 


degree of 


flexibility in what can be 


done is not 


great and 



certainly not as great as wa3 apparent in the food service. In retro* 



Spect, ie would have been wiser to do the phirmacy study first anJ the 
food service study second. 

It is important to note, however, that in both interviews the 
respondents proved most interested in the study, tn the case of the 
pharmacy the respondent asked to keep copies of the descriptions of 
input and output feeling that this would be helpful in making budgetary 
requests. H« felt they provided descriptions that vould bo useful in 
budget requests. Vie also consented that when making choices he realised 
t.iat th*3e were Certainly tn« Lind of choices he was always ma ting. 







The figures in Table III with asterisks correspond to figures 
that would imvc been secured using the technique used in the Fooi 
Service studyi 

Third Interview to Identify Relationships 

The final interview was designed to secure directly, information on 

the relative distance between the five points on the output scj1g> 

The technique is described in detail in an article by Sidney Siegel, 

9 

"A Method for Obtaining in Ordered Metric Scale.' Briefly, the re- 
spondent was given a aeries of cards on which he had to oake a choice 
between two pairs of the five points on the ucalc expressed in a two- 
by-tvo oatrix. Ho wao told that after he had selected one of the two 
pairs a single itcti in the pair would lu selected with each having the 
sane probability! bach card had the following forent: 



50 

50 



A L 



1 


II 


IV 


III 



The respondent was asked to pick either A or B and was told that 
the upper or lover rov would then be selected for hi:* with each row having 
n probability of .5 of being selected. For exanpla, in this illustra- 
tion, if he Selected A then he knew either 1 or IV vould be selected 
with a probability of .5. 

In aaking up the cards all possible conbinatlons of the five 
points were prepared following tho rule that the two ltces in the 



colls on the right would always fall between the two Items on the lift. 
A total of 13 cards were prepared. 

The results were Interpreted In the following manner. In the 
illustration above! for example, If the respondent chose A, It va3 
assumed that IV and III were closer then I and II. Fut somewhat 
differently, the option of A was preferred to B because the Rain In 
I over II va 9 greater than the lo3s of IV compared with III. This 
can be readily seen if instead of I, II, III, and IV, wo had In tho 
four cells $luO, $15, $10, and $5 In tne same order. 

Oii the other hand, if the choice wore B, it would be assumed that 
tiie distance between IV and III were greater than I and II. Thl3 can 
be seen if instead of I, II, III, and IV, wu had in tho four cells 
$100, $15, $13, and -$500. The prospect of paying $500 might be 
sufficiently bad to cau3e the respondent to choose B with the choices 
of $15 and $10. 

Using this technique and following closely tho procedure suggested 
by Siegel, it was discovered that tho respondent 1 s choices were per- 
fectly transitive and consistent in all 13 cards. This than gave an 
ordered Metric measure for the five points as shown below where the 
distancc^were a3 follows: 

IV to V greater than 111 to IV 
111 to IV greater than 11 to 111 
It to 111 greater than 1 to It 

Graphically this is given below: 

1 11 m IV V 

Combining this information with that received in interview number 
two would suggest that the marginal return following the achievement of 






level III started decreasing rabidly. After achieving Level III with 
budget C-, increases the budget needed to achieve Level I which possi- 
bly could nave been done vita a budget less than A, did not he\e as 
great a payoff as did the cruller budgets. This information certainly is 
consistent with the concept of diminishing returns. In this case, 
they begin to set in sooner than in the case of the 

food service which again is reasonable since there is greater flexi- 
bility in the use of resources there than in the operation of a small 
pharmacy. 

Chart V has been drawn to correspond with Chart IV. The five 
points on the Y axis have been placed to reflect the respondent's 
ranking of distance between the five points. Tne items in Table III 
witti asterisk are the ones plotted. ! 

C0..CLUS10.I 

In neither of the t'/o cases was there a problem in securing reason- 
able descriptions. In the interviewing situations there was 9oma con- 
fusion caused by using the word 'level" in discussing inputs and outputs. 
Sometimes the respondent would think that level I input would give 
level 1 output. It would have been better to use sooe tern other than 
level" for either input or output. One could have simply referred 
to output as A, £» C, D, and E. It might also have been better to 
reverse the order so that increasing amounts of input were reflected 
by going from IV to 1 while increased levels of output eight have been 
reflected by going from A to £. 
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While these two exauplo3 did not take Into consideration signifi- 
cant fluctuations in demand on the systems, the descriptions could be 
prepared to reflect this where demand can vary greatly. The highest 
level could indicate the availability of resources to allow the system 
to perform at the top level at peak demand. Another variable not in- 
cluded in the two tests was the possibility of comparing different 
technologies or systems but these two variables could also be incor- 
porated in the technique. 

An Interesting problem arose during the interviewing for the 
food service. To secure some additional experience working with the 
technique the administrator directly over the person interviewed was 
also interviewed. This respondent was given the same questionnaire 
to select the preferred input configuration for each budget and lie 
completed the questionnaire indicating his choice. A limited inter- 
view was also conducted in which he was asked to express his choice 
between a 100*. chance at receiving a given budget configuration, or a 
502 chance at achieving a given level. This was the same technique 
used in interviewing the individual directly In charge of the food 
service. The question raise! by the respondent in the interview 
was: "If 1 select the choice giving do a 502 chance at Level 1, what 
happens if 1 don't achieve level? toes 'a 502 change at level I mean 1 
could get something a little less than 1 or much less than 1." 

To one not acquainted vitn the use of probabilities in a gamin? 
situation to find points of indifference, an alternative might be 
used where the individual is given a two-by-tvo matrix as was done 
vita the administrator of the pharmacy in comparing tne relative 



-jo - 



values of the five levels of output. Some have expressed that this 
Is a better approach to measuring utility. (This is the approach 
suggested by Frank Ramsey. Kellner discusses the Issue briefly in his 
book Probability and Profit .^ ) In this situation the use of the 
technique would be to place In the two cells on the left output levels 
and on the right input budget configurations. This was not done In 
this study because the results .achieved were considered satisfactory 
in testing the basic approach to naklng explicit assumptions about 
the relationship between resource Inputs. Using such cards night, 
however, secure more satisfactory lnfornatlon. 

It should be repeated for emphasis that the technique seeks to 
nake explicit a person's assumptions about the relationship between 
Input and output. It does not validate the relationships expressed. 

In using the technique It Is also necessary to ronenber that a person's 
aversion to risks is an Important factor and should be measured as 
an aid in analysing the results of measures suggested here. An Inter- 
esting procedure would be to use this technique to nake explicit the 
opinion of several people involved in a progran and then use this 
material as a basis of a conference to develop a group consensus on 
the relationship. Since group consensuses are used to allocate re- 
sources, these techniques could help make the thinking of nonbers of 
the group nore explicit. Another Interesting possibility would be 
for a study to begin by developing the output scale based not on an 
Interview with those responsible for a prograa but rather those who 
are Its consumers. The interview would seek to Identify those eaplrlcal 
criteria used by the consumers of a program's output to evaluate 




tho program. One could then Interview those responsible for the 

program to relate Lngel cuive points to the output scale based on 

'Using 

consumers. There would of course be a serious problem in ona* this 
technique if those responsible for the program were unable to relato 

the points on the Engel curve to the consumers output scale because 

re ferents 

it was so different from one ba&:d on their AMnCTfor output. 

A final point to stress is the need to develop detailed and 
empirical descriptions of inputs and especially outputs and within 
the framework of the respondent. This provides a sound basi6 for 
analyzing co9t data and assumptions. In these two teats the major 
portion of the total time was devoted to this task. 
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APPE.UIX A 



Program: Hospital Food Service 

Output (Performance) 

Level I 



Aspect I, ilENU ; 

The patient can select from a generous variety of meat, vegetables, 
salads, desserts, breads, and beverages* 

The menu includes high-quality tender cuts of meat (some cooked 
to order), gourmet delicacies such as gam e, lobster tail and shrimp 
Wewburg; and desserts contain generous portions of expensive ingre- 
dients such as butter, eggs, and flavorings (for example, French 
ice cream) . 

Preparation is excellent, the food is very delicious, and the 
portions are generous* 



Aspect 2, PERSONAL ATTEWTIOU ; 

Heals are served within specified hours; but during that time the 
patient rings when he wants his meal, someone comes to him to take 
his order, and the main course is served promptly. 

When the patient finishes his main course, he rings, and someone 
comes to remove dishes and take his order for dessert* Dessert is 
served immediately. 

When the patient finishes dessert, he rings again, and the re- 
mainder of the meal is removed promptly* 

Someone checks frequently to see that food and service are 
satisfactory. 



Aspect 3, SERVICE DETAILS : 

The food is served at proper temperature almost without 
exception. 

The tray is attractively arranged with good quality ceramic 
d. 3hes, good quality flatware and glassware, and cloth napkins. 
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APPENDIX A 



Program: hospital Food Service 



Output (Performance) 



Level II 



Aspect 1, liENU ? 

The patient can select from a more restricted list of meats, 
vegetables, and desserts than ir: Level I service. 

The menu includes no gourmet delicacies or dishes requiring 
individual preparation, but food is of good quality. It includes less 
expensive types of meat (as some of the "tough 1 ' cuts) and desserts 
(for example, good quality American ice cream.) 

Preparation is good, the food is appetising, and portions are 
adequate for most tastes. 



Aspect 2, PERSONAL ATTENTION : 

Teals art served to all patients at one time, and service is prompt. 
The patients indicate their selections on a card at the previous meal. 

Lessert is served 15 minutes after the main course, at vrhich 
time the main course dishes are removed. 

Trays are collected from all patients at one time. 

Someone checks occasionally to see that food and service 
are satisfactory. 



Aspect 3, SERVICE DETAILS : 

The food is served at proper temperature 90/1 of the tine. 

The tray is neat, and individual ceramic dishes are used. Nap- 
kins are of good quality paper. 
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Program: Hospital Food Service 

Output (Performance) 

Level III 



Aspect 1, rlEijU ; 

The institution uses many menus, but the patient has no choice 
at any one meal. 

Food is average in quality and includes a high proportion of 
"budget' 1 foods such as hamburger dishes, chili, and popularly priced 
ice cream. 

Preparation is average, the food is palatable, and portions are 
adequate . 



Aspect 2, PERSONAL AIM ION : 

Heals are served to all persons at one time, and service is 
prompt . 

Dessert is served on the tray with other courses. 

Trays are collected from all patients at one time. 

Very seldom is any check made to 3ee that food and service are 
satisfactory. 



Aspect 3, SERVICE DETAILS : 

The food is served at proper temperature 2/3 of the time. 

Food is served in individual dishes. 

Little attempt is made to arrange the tray, but paper napkins 
are supplied. Appointments tend to 'look cheap." 
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APPENDIX A 



Program: Hospital Food Service 

Output (Performance) 

Level IV 



Aspect 1, MENU : 

The same menu is offered every 7 days, and the patient has no 
choice at any one meal. 

Food is inferior in quality and preparation, and is unappetizing. 
Heat portions contain large proportions of fat and gristle, potatoes 
are often blackened and waterlogged or soggy with grease; vegetables 
are "old 11 or overcooked; the rice pudding is gluey or baked to a 
crisp; the pie crust is tough, thick and indigestible. 



Aspect 2, PERSONAL ATTENTION : 

Meal times vary as much as an hour. 

The entire meal is served at once. 

Trays are seldom collected less than 1-1/2 hours later. 



Aspect 3, SERVICE DET AILS: 

The food is rarely served at proper temperature. 

Food is served in metal compartment trays and is sloppily 
arranged. 

No napkins are supplied. 
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APPENDIX 3 



OUTPUT: Drug Service 

ASPECT: Formulary 



LEVEL I: The Health Service Physician has the option to pre- 

scribe the exact medication he wishes the patient to 
have, (I.e. virtually all types and concentrations of 
medications are stocked.) 

dedications and therapeutic equipment are dispensed 
free of charge to students and other university 
personnel. 

Students are able to obtain common medicinals such as: 
aspirin, salt tablets, vitamin pills, anti-bacterial 
soap, etc., free of charge from the pharmacy upon 
request. (Ho prescription is necessary.) 



LEVEL II: The student may receive precise medication for those 

diseases previously anticipated by the Therapeutics 
Committee of Health Service* 

The major portion of medication is dispensed free of 
charge to students. The patient must, however, pay 
cost price for the very expensive drugs. 

Students are able to obtain common medicinals such 
as; aspirin, salt tablets, etc., upon request. They 
must, however, pay cost price of the Item. 

LEVEL 111: The student Is able to obtain medication for 'chronic 1 

diseases from the Health Service Tharmacy. Medication 
for other diseases and Infections must be obtained 
elsewhere. 



LEVEL IV: 



LEVEL V: 




Students must pay cost price for all medication re- 
ceived from the pharmacy. 

They can only receive medicinals via prescription and 
must pay retail price for these. 

There Is a very limited supply of drugs which the 
student may obtain from the Health Service Pharmacy. 

Students must pay retail price of those received. 

l!o medicinals are available for students. 

No pharmacy per se. 

ho drugs or medicinals are dispensed from Health Ser- 
vice Pharmacy. A few aspirin, cold tablets are avail- 
able In the physicians 1 offices. 
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APPENDIX B 



OUTPUT: Drug Service 

ASPECT: Service Details 



LEVEL It Students receive the exact medication prescribed upon 

arriving at Health Service Pharmacy in convenient 
"pop top" plastic containers. Doses are individually 
packed inside the container. 

dedication is dispensed to all areas (hospital, out- 
patients, emergency clinic, etc.) at all hours of the 
day and night - seven days a week. 

The attitude of the Pharmacy personnel is very friendly 
and professional. Verbal instructions are given in 
addition to the typed label. Uhere antibiotics are 
prescribed the patient is given the first dose on 
the spot. 

Drugs are delivered to the student's residence twice 
daily. 

LEVEL II: Students receive the prescribed mediation from Health 

Service Pharmacy In convenient rigid containers with 
minimum delay. 

Medication is dispensed 12 hours a day - six days a 
week. The needs for other areas such as emergency 
room and hospital during the closed hours are accur- 
ately predicted and stocked each evening. 

The Pharmacy personnel are quite amiable. Instructions 
are typed clearly on the package. 

Deliveries are made to the student's residence each 
day around supper time. 

LEVEL III: Students receive the prescribed medication from H.S, 

Pharmacy in flimsy paper containers. Students must 
wait in line to receive their prescription. The 
length of delay is a function of the number of stu- 
dents requiring medication at this particular time, 
(usually about 10-20 minutes delay.) 

Students may have their prescriptions filled eight hours 
a day, five and one-half days a week. Other areas such 
as Hospital and Emergency Room must anticipate their 
needs daily and request these from the Pharmacy each 
morning. 
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LEVEL 



LEVEL 



APPEiim Q 



(continued) 



OUTPUT: Drug Service 

ASPECT: Service Details 



IV: Medication is dispensed each morning from 9 a. a. 

to 12 p.m., Monday through Friday. As a result, 
long lines usually develop and both patient and 
pharmacist become quite irritated. 

Ho medication other than aspirin, etc. , is available 
to other areas of health service during closed hours. 

V: Health Service does not maintain a Pharmacy per se 

which dispenses drugs. What drugs are available are 
stored in the physicians’ off ices and given to the 
student where possible. 



APPKJUIX D 



OUTPUT: Drug Service 

ASPECT: Information Service 



LEVEL I: 



LEVEL II: 



LEVEL III: 



LEVEL IV: 



LEVEL V: 



Virtually any type and amount of information 
concerning drugs and their manufacturer's is 
available (to physicians or the university com- 
munity) immediately upon request of the Health 
Service Pharmacy. The information given is 
continually being updated and includes informa- 
tion ou both medical drugs and other chemicals 
of interest such as L.S.D., etc. 

The information available from Health Service 
Pharmacy covers a bro3d but non-intensive area 
of drugs and their manufacturers. 

The information available from Health Service 
Pharmacy is limited to the general knowledge 
of drugs obtained by Pharmacists during their 
university training as students of Pharmacology. 

Very little information concerning drugs is 
available to the university community. Ho infor- 
mation concerning drug manufacturers is available. 

Ho information is available. 
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Amaru ix ti 



OUTPUT; Physical Atmosphere 
ASPECT; Pharnacy Appearance 



LEVEL Is 



LEVEL II; 



LEVEL Ills 



LEVEL IV j 



LEVEL V: 



The Health Service Pharmacy is housed in a veil 
lit* fully enclosed area. Sound-proof glass and 
wood panels separate the dispensary area from the 
library and reference area as well as the research 
area, etc. The design of the department layout 
and the specific activity areas provide for optimal 
efficiency of effectiveness. All of the phar- 
macy *s business activities (cash budgets, receipts, 
etc.) are centrally administered by the Health 
Service Business lanagor. 

The Health Service Pharmacy is located in fully 
enclosed area. This area provides adequate space to 
enasle certain activities to be separated in terms of 
their location. (There are no physical partitions 
separating the activities.) Pharmacists must handle 
all business activities such as balancing cash re- 
ceipts with register slips, etc. 

The Health Service Pharmacy is located in a snail room. 
Adequate space exist3 for the present limited line 
of drugs only. All activities (dispensary, re- 
ference material, compounding, cash register, etc.) 
are located within the same area. This causes some 
distraction. Aisles are tight and there is little 
room for expansion. 

Because of limited available space, the Health Ser- 
vice Pharmacy must share its quarters with two other 
departments (student medical record department and 
emergency treatment room. ) The area is cramped and 
much confusion and distraction results. 

\lo pharmacy per se - the small amount of drugs 
available are stored in the physicians 1 offices. 




APPENDIX 3 



LEVEL I: 



LEVEL II; 



LEVEL III: 



LEVEL IV; 



LEVEL V: 



^Stockouts : 



OUTPUT: Physical Atmosphere 

ASPECT : Professionalism 



Drugs in open stock are neatly arranged and classi- 
fied in the most effective manner. New shelf arrange- 
ments are introduced at varying interval s to assure 
proper checking and control with regard to filling 
prescriptions. Dispensing procedures employed by 
the Pharmacists arr tnc best available and provide 
for minimum dispensing time with maximum effective- 
ness and control. The inventory control system is 
very effective. Stockouts are never experienced.* 

Drugs in open stock are well arranged and classified 
adequately. Lew shelf arrangements are introduced 
only after a mistake in filling out a prescription 
is made and has been detected. Dispensing and in- 
ventory control procedures are for the most part ade- 
quate. Stockouts occur, but very infrequently. 

The arrangement of open stock drugs provides minimum 
effectiveness. The Inventory and Dispensing Procedures 
employed do not provide the speed and control necessary 
for effective and efficient operations. Students are 
frequently required to wait 10 to 15 minutes for their 
prescriptions to be serviced. 

Because of the cramped quarters the open stock drug 
arrangement is confusing. A rudimentary inventory 
system exists. Stockouts occur frequently. 

No pharmacy per so - dispensing procedures are subject 
to the desires of the individual physicians. 



running out of stock for a particular drug. 



